Conventional image coding processes are broadly divided into two of coding processes performed in block units, represented by MPEG2-based coding methods, and coding processes performed in pixel units, such as differential pulse code modulation (DPCM: Differential Pulse Code Modulation).
The coding process in block units is a method in which a single image display region is divided into plural blocks and a coding process for an image signal that is input (hereinafter referred to as an input image signal) is performed block by block. In this case, the single image display region corresponds to a single display screen in the MPEG2-based coding process, and in an MPEG4-based coding process, it corresponds to a display region having a shape and a size corresponding to each object on a single display screen. Further, each of the above-mentioned blocks is a display region composed of a prescribed number of pixels within the single image display region, and a rectangle shape which is easily processed is used as the shape of the block in many cases.
As described above, in the coding method in which a coding process for an input image signal is performed block by block, the coding process for the input image signal corresponding to a single image display region is completed in each block. Therefore, there is the advantage that, even though a transmission error occurs when a coded image signal obtained by performing the coding process for the input image signal is transmitted, influence of the error can be converged in each block.
However, the block-by-block coding method has the following drawbacks.
Since the coding process for the input image signal is completed in each block in the block-by-block coding method, it is difficult to use an inter-block pixel correlation, that is, a correlation of pixel values which are present in different blocks, in the coding process.
Further, in a predictive coding method for an image signal, a pixel value of a coding target pixel being a target of coding (a coding target pixel value) is predicted with reference to pixel values of plural coded pixels which have previously been coded (coded pixel values), and the coding target pixel value is adaptively coded using the predicted pixel value. In this predictive coding method, however, when the coding process is performed block by block, the coded pixel values to be referred to when coding the coding target pixel value are limited to pixels within each block, so that the reference coded pixel values are small in number. Therefore, the accuracy of the predicted value of the coding target pixel is reduced, and the coding efficiency is not increased very much.
On the other hand, the coding method in pixel units is a method in which an input image signal is coded pixel by pixel, and in this coding method, it is possible to change the coding process for the input image signal pixel by pixel. Therefore, when this coding method includes a universal coding process, such as adaptive arithmetic coding in which code words are automatically updated pixel by pixel adaptively to the characteristics of the input image signal, an image signal having any characteristic can be coded with a significantly high coding efficiency.
However, since, on the decoding side, a coded image signal obtained by the pixel-by-pixel coding method including the universal coding process is subjected to a decoding process in which code words are updated in the same manner as on the coding side, when a transmission error occurs when the coded image signal is transmitted, the state in which the decoding process for the coded image signal cannot be carried out accurately because of the influence of the transmission error on the decoding side, continues long.
By the way, the block-by-block coding method and the pixel-by-pixel coding method can be combined, and in a coding method in which these coding methods are combined (hereinafter referring to this coding method as a combination coding method for explanation), code words can be adaptively changed for each pixel, and the influence of transmission error can be converged in each block, whereby a coding process with a high coding efficiency, such as adaptive arithmetic coding, can be performed while suppressing the influence of transmission error.
A description is now given of this combination coding method.
FIG. 13(a) shows the state in which a single frame screen is divided into a plurality of rectangle blocks, and FIG. 13(b) shows arrangement of pixels in blocks, especially in a coding target block being a target of coding and blocks in the vicinity of the coding target block. Needless to say, these pixels are arranged in matrix along horizontal scanning lines in the single frame screen.
In the figures, FG denotes a screen corresponding to a single frame, B1 denotes a coded block in which a coding process for an image signal has already been performed, Bx denotes a coding target block being a target of coding, and B0 denotes an uncoded block in which a coding process for an image signal has not been performed. When no distinction is made between blocks, blocks are denoted by B. BLu, BLs, BLh, and BLm denote upper, lower, left, and right boundaries of the coding target block in the single frame screen, respectively. A solid line circle shows a coded pixel whose pixel value has already been coded, and a dotted line circle shows an uncoded pixel whose pixel value has not been coded yet. Each block B is an image display region comprising 4.times.4 pixels, in the single frame screen FG.
FIG. 14 shows positional relationships between a coding target pixel Px to be coded and peripheral pixels P0.about.P9 surrounding the coding target pixel, and the pixel values of these peripheral pixels P0.about.P9 are referred to when the pixel value of the coding target pixel Px is predicted, so that these pixels are called reference pixels hereinafter.
The reference pixels P8 and P9 are pixels which are positioned in the same horizontal scanning line as the coding target pixel Px, and the reference pixels P9 and P8 are positioned one pixel and two pixels before the coding target pixel Px, respectively. The positions of the reference pixels P5 and P1 in the horizontal direction on the single frame screen FG correspond to the position of the coding target pixel Px, and the reference pixels P5 and P1 are positioned in a horizontal scanning line by one pixel and two pixels upper than the coding target pixel Px, respectively. Further, the reference pixels P3, P4, P6, and P7 are pixels which are positioned in the same horizontal scanning line as the reference pixel P5, the reference pixels P4 and P3 are positioned one pixel and two pixels before the coding target pixel Px, respectively, and the reference pixels P6 and P7 are positioned one pixel and two pixels after the coding target pixel Px, respectively. Further, the reference pixels P0 and P2 are pixels which are positioned in the same horizontal scanning line as the reference pixel P1, the reference pixel P0 is positioned one pixel before the reference pixel P1, and the reference pixel P2 is positioned one pixel after the reference pixel P1.
In the combination coding method, initially, an image signal corresponding to the single frame screen FG is divided correspondingly to plural blocks B constituting the single frame screen as shown in FIG. 13(a) and FIG. 13(b), and a coding process for the divided image signal is performed block by block.
This block-by-block coding process is completed by performing a horizontal process in which pixel values of pixels are successively coded from the left to the right along each horizontal pixel line in a block B, for all of the horizontal pixel lines in the block from the uppermost line to the lower most line.
In this coding process, as shown in FIG. 14, the pixel value of the coding target pixel Px is adaptively predicted from the pixel values of the reference pixels P0.about.P9 positioned in the vicinity of the coding target pixel, and code words used for the coding process of the coding target pixel are adaptively selected according to a predicted value obtained by the prediction.
Therefore, in the combination coding method, the influence of transmission error on the decoding side can be converged in each block, and the coding efficiency can be improved as compared with the simple block-by-block coding process.
FIG. 16(a), FIG. 16(b), and FIG. 17 are diagrams for explaining a combination decoding method corresponding to the above-mentioned combination coding method, and in the figures, B' denotes each block in a single frame, Bx' denotes a decoding target block, B1' denotes an already decoded block, B0' denotes an undecoded block, BLu', BLs', BLh', and Blm' denote block boundaries of the decoding target block Bx' at the upper, lower, left, and right sides, respectively, and P0'.about.P9' denote reference pixels corresponding to a decoding target pixel Px'. In this case, the arrangement of the reference pixels P0'.about.P9' with respect to the decoding target pixel Px' is identical to that described for the coding process shown in FIG. 13(a), FIG. 13(b), and FIG. 14.
In the combination decoding method, initially, an image signal corresponding to a single frame screen FG' is divided correspondingly to plural blocks B' constituting the single frame screen as shown in FIG. 16(a) and FIG. 16(b), and a decoding process for the divided image signal is performed block by block.
This block-by-block decoding process is completed by performing a horizontal process in which pixel values of pixels are successively decoded from the left to the right along each horizontal pixel line in a block B', for all of the horizontal pixel lines in the block from the uppermost line to the lower most line.
In this decoding process, as shown in FIG. 17, the pixel value of the decoding target pixel Px' is adaptively predicted from the pixel values of the reference pixels P0'.about.P9' positioned in the vicinity of the decoding target pixel, and code words used for the decoding process of the decoding target pixel Px' are adaptively selected according to a predicted value obtained by the prediction.
However, the combination coding method in which the block-by-block coding process and the pixel-by-pixel coding process are combined has the following drawbacks.
In this combination coding method, since the coding process proceeds block by block, when the coding target pixel Px abuts the right boundary BLm of the coding target block Bx as shown in FIG. 15, the reference pixels P2, P6, and P7 corresponding to the coding target pixel Px are uncoded pixels.
In this case, when the pixel value of the coding target pixel Px is predicted with reference to the pixel values of the uncoded pixels P2, P6, and P7 and the pixel value of the coding target pixel Px is coded using the predicted value, a coded image signal corresponding to this coding target pixel Px cannot be accurately decoded on the decoding side. More specifically, in order to accurately decode a coded image signal which is obtained by coding the pixel value of the coding target pixel Px using the predicted value, the predicted value of the decoding target pixel Px' used in the decoding process must agree with the predicted value of the coding target pixel Px corresponding to the decoding target pixel Px', used in the coding process. In other words, on the coding side, the reference pixel values referred to for generating the predicted value of the coding target pixel Px must completely agree with the reference pixel values referred to for generating the predicted value of the decoding target pixel Px' corresponding to the coding target pixel Px.
For this reason, for example, as shown in FIG. 15, when the coding process for the coding target pixel Px is performed, if the predicted value of the coding target pixel Px is generated with reference to the pixel values of the uncoded pixels P2, P6, and P7 among the reference pixels P0.about.P9 corresponding to the decoding target pixel Px, as shown in FIG. 18, when the decoding process for the decoding target pixel Px' is performed on the decoding side, the predicted value of the decoding target pixel Px' is generated with reference to the pixel values of the reference pixels P0'.about.P9' corresponding to the decoding target pixel Px', but among the reference pixels P0'.about.P9', pixel values of the undecoded pixels P2', P6', and P7' are not obtained on the decoding side, so that the pixel value of the decoding target pixel Px' corresponding to the coding target pixel Px cannot be decoded.
Therefore, in the conventional combination coding method, in order to avoid the problem that the decoding process becomes difficult when uncoded pixels are included in the reference pixels P0.about.P9 corresponding to the coding target pixel Px as described above, there is taken a countermeasure in which the predicted value of the coding target pixel Px is generated with the pixel values of the uncoded pixels being regarded as a fixed value which has been previously set (e.g., 0), and the coding process for the coding target pixel Px is performed using this predicted value.
Although the combination coding method with the above-mentioned countermeasure enables the decoding process to be accurately performed for all of the pixels in each block using their predicted values, since the pixel values of the reference pixels being uncoded pixels are uniformly replaced with a fixed value, the correlation of pixel values between the uncoded pixel and the coded pixel is degraded, resulting in the problem that the efficiency in predicting the coding target pixel, i.e., the accuracy of the predicted value of the coding target pixel, is degraded.
The present invention is subjected to solving the above-described problems, and has an object to provide an image processing apparatus and an image processing method, which can combine an adaptive pixel-by-pixel coding process and a block-by-block coding process, without degrading correlation of pixel values between uncoded pixels and coded pixels, with avoiding that decoding of a coded image signal becomes difficult, and to provide a data recording medium in which an image processing program for realizing the image processing method is recorded.
It is another object of the present invention to provide an image processing apparatus and an image processing method, which can perform an accurate decoding process for a coded image signal which has been coded, without degrading an efficiency in predicting coding target pixels, and to provide a data recording medium in which an image processing program for realizing the image processing method is recorded.